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FUN NUMBERS FIRST: 
LAUNCH RESULTS 


• Events/Second 

— Averaged 701k/s (2-5B/hr) 

- Peak fl31k/s 

• Bad Events 

— Composer event MMbk/sCpeak second) 

— Post Game Update TOk/s (peak second) 


OVERVIEW 


• Where we started 

• Goals 

• Implementation 

• Results 


WHERE UE STARTED: LOG EVENT MACRO 


LOG_MESSAGE(EC_Users > "PlatformEventManager - User Engaged! ActiveUserlndex: %d %I64u", userlndex, xuid); 

• Legacy ‘•DataMine ' 1 system 

• Good: 

— Easy to use 

• Bad : 

— No strong typing 

• Unenforced parameter naming 
— Transmits zipped strings at end of match 
— Slow - not compiled in Release 


WHERE WE 

• Binary Lo< 

• Good 

— More com 
— Easy dev 

• Bad 

— No share 
— Requires 
— T ransmit 


STARTED: BINARY LOG 


3 Format 


pressed 

use 


D // this is the player-chosen loadout definition 
// blf: PlayerLoadoutSlot 
// blf: PlayerProfilelnterface 
3 struct s_custom_loadout_slot 
{ 

D // these are indices into the appropriate global multiplayer object arrays, the 
II string id for the object referenced needs to be derived from this, 
long primary_weapon_index; 
long secondary_weapon_index; 
byte primary_weapon_variant_indexj 
byte secondary_weapon_variant_index; 
byte pad[2]; 
long equipment_index; 
long grenade_index; 


void initializeCustomLoadoutSlot(int slotlndex); 

}; 

COMPILE_CHECK_SIZE(s_custom_loadout_slot, 20); 


d schema: Nightmare for services 


source to understand 


s at end of match 


“GAMING INTELLIGENCE 11 

• The methods and technologies that gathers 
storei reportn and analyze game data to 
help people to understand and improve the 
game • 


AIMING HIGHER 


Reliably transmit telemetry in 
real-timen enabling subsecond 
service response- 


Maelstrom 



Client Message Pump 
(16ms) 



Queue/Dequeue 

(50ms) 


■< 


Queue/Dequeue 

(50ms) 


J 




V_ 



J 




AIMING HIGHER 


1. Sequential 
S- Contextual 
3- Compatible 


AIMING HIGHER 


1- Sequential 
B« Contextual 
3- Compatible 


AIMING HIGHER 


1- Sequential 

2- Contextual 
3* Compatible 


AIMING HIGHER: CELL AND MAELSTROM 

Common Event Logging Library (CELL) 

• Telemetry Manager 

• C + + Macro/Funct ion (1S-3Dijs) 







CELL AND MAELSTROM 


Common Event Logging Library (CELL) 

• Telemetry Manager 

• C + + Macro/Funct ion (1S-3 Dvls) 

• Build-time preprocessor 

- Global Schemastore 

- “Ever Growing Schema 11 : Forwards & 
Backwards Compatible 

- BOND ser ial ization (typical events ~1ED 
bytes ) 


CELL AND MAELSTROM 


Maelstrom 

• Ingestion Service 

• BOND wrappers 

• Azure Event Hub 


MAELSTROM CONSUMERS 


• Common API for consuming event 
streams • 

• Orleans (distributed virtual actors) 
— Stats 

— Storage 

• Librarian (Session/Blob Indexer) 

• Telemetry Event Viewer ETEVJ 


HADOOP / HIVE 


• Hadoop/HIVE 

— Custom Java BOND SerDe reads raw 
BLOBi stores into ORC 

— Regular jobs run hourly for BI 
reports ■ 

- Ad-Hoc reports in H<2L/Tableau/R 


GOTCHAS 


• Client implementation took time 

• Statistics Event: Nonsequential 

• Hibernation makes for very long 
sessions 

• Stream Querying deferred to post- 
launch 

• Schema Store caused problem during 
iteration 


RESULTS: HAP BALANCE 


Rocket FFA 

Stayer 

SWAT 

Team Arena 


FFARockets 


Slayer 

SWAT 

SWATnums 

Slayer 

Strongholds 



Acropolis Remix Rocket FFA FFARockets 

47.57% 

54 55% 

50 00% 

56.84% 


Slayer Slayer 

56.95% 

55.00% 

55.17% 

5540% 

55 84% 

SWAT SWAT 

SWATnums 

5645% 
57 17% 

60.92% 

60.36% 

59.57% 
58 86% 

60.29% 

57.97% 

58 07% 

60 37% 

Team Arena Slayer 

Strongholds 

58.17% 

55 80% 

55 34% 

52.88% 

53 01% 
53 82% 

53 88% 

53.66% 

51 58% 

55.88% 


BlueTeamMMRDeltaBinned / Team Color 




-2 

-1 

-0.9 

-0.8 

-0.7 

-0.6 

-0.5 

-0.4 

-0.3 

-0.2 

-0.1 

0 

0.1 

0.2 

0.3 

0.4 

0.5 

0.6 

0.7 

0.8 

0.9 

1 

Piaylistname 

Gamevariantname 

Blue 

Blue 

Blue 

Blue 

Blue 

Blue 

Blue 

Blue 

Blue 

Blue 

Blue 

Blue 

Blue 

Blue 

Blue 

Blue 

Blue 

Blue 

Blue 

Blue 

Blue 

Blue 

Rocket FFA 

FFARockets 




44.44% 

3636% 

KjTjT] 

44.44% 

45.78% 

44.44% 

44.86% 

49.82% 

50.77% 

51.27% 

50.89% 

43.97% 

48.94% 

58.82% 

55.26% 

44.44% 




Slayer 

Slayer 


4.76% 

7.97% 

6.57% 

1471% 

1805% 

26 82% 

29 46% 

42.66% 

45.84% 

50.62% 

55.62% 

62.99% 

66.35% 

70.21% 

77.89% 

81.74% 

90.79% 

91.19% 

92.42% 

97.33% 

| 97.30% 

SWAT 

SWAT 



6.90% 

2143% 

35 80% 

35 83% 

38 13% 

35 90% 

46.97% 

51 01% 

55 16% 

59 19% 

66.54% 

69.40% 

7320% 

81.55% 

84.67% 

88 97% 

92.31% 

94 92% 

82.14% 



SWATnums 



1786% 

23 26% 

1848% 

2542% 

37 43% 

33 49% 

45.71% 

52.31% 

54.81% 

58 95% 

63 28% 

69 42% 

70 00% 

78.24% 

83.33% 

89 83% 

87.84% 

84 62% 

83 72% 


Team Arena 

Slayer 



5.00% 

10 59% 

1259% 

15.47% 

2697% 

28 36% 

41 16% 

4519% 

49.57% 

55.20% 

64.01% 

65 28% 

65.32% 

83 09% 

76.67% 

91.01% 

93.29% 

96.77% 

100 00% 



Strongholds 



1316% 

1477% 

13 68% 

24 20% 

31 98% 

30 46% 

43.81% 

42.88% 

47.32% 

53.66% 

59.48% 

59 88% 

60 04% 

79.51% 

77.22% 

82.80% 

8681% 

88.06% 

97 26% 















RESULTS: DEATHS 
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RESULTS: TRAVERSAL 




mini 


RESULTS: LETHALITY 
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RESULTS: HEATNAPS 


® B *5 • c* • • 

I HOME IN 

gk <^>cut 

fT[=) Copy ’ 

Paste 

. ^ Format Painter 

Clipboard fi 


PegasusLocations.xIsx - Excel 

DATA REVIEW VIEW LOAD TEST POWER QUERY POWERPIVOT TEAM 


Calibri - |l1 - A' a' “ =|= 

’ gf' Wrap Text General 

m 9 

Normal 

Bad 

Good 

B I u - 03 - 2 j-A- - - = 

[H Merge & Center ’ $ * % * 

«-.o .oo Conditional Format as 
.00 -*.o 

Formatting’ Table’ 

| Calculation 

□ ISES 

| Exp tonal 






Font ri 

Alignment Number 

fi 


Styles 




M N 


RED SPAWN 


RED SPAWN 
RED SPAWN 
RED SPAWN 
RED SPAWN 
RED SPAWN 
RED SPAWN 
RED SPAWN 
RED SPAWN 
RED SPAWN 
RED SPAWN 
RED SPAWN 
RED SPAWN 
RED SPAWN 
RED SPAWN 
RED SPAWN 
RED SPAWN 
RED SPAWN 
RED SPAWN 
RED SPAWN 
RED SPAWN 
RED SPAWN 
RED SPAWN 
RED SPAWN 
RED SPAWN 
RED SPAWN 
RED SPAWN 
RED SPAWN 
RED SPAWN 
RED SPAWN 
RED SPAWN 


-31.3519 y 
-31.6853 y 
-31.9903 y 
-32.0481 y 
-31.9688 y 
-32.2031 y 
-32.0067 y 
-31.7172 y 
-31.4299 y 
-31.1603 y 
-30.8412 y 
-30.3218 y 
-29.8558 y 
-29.1384 y 
-28.8248 y 
-28.0571 y 
-27.61 y 
-26.8518 y 
-26.2377 y 
-26.2641 y 
-26.2587 y 
-26.2412 y 
-26.0903 y 
-25.5725 y 
-25.5345 y 
-25.5173 y 
-25.4085 y 
-25.3343 y 
-25.3267 y 
-25.4371 y 
-25.5187 y 


25.3094 z 
24.395 z 
23.5564 z 
23.0853 z 
22.8008 z 
22.2009 z 
21.788 z 
21.1703 z 
20.5478 z 
19.9552 z 
19.7675 z 
19.2852 z 
18.918 z 
18.6774 z 
18.7015 z 
18.5198 z 
18.1383 z 
18.1392 z 
18.0536 z 
18.4164 z 
19.2213 z 
19.9192 z 
20.1283 z 
20.1166 z 
20.1083 z 
20.5824 z 
21.1993 z 
21.8126 z 
22.5297 z 
23.1949 z 
23.1458 z 


-8.18293 

-8.18611 

-8.15238 

-8.15697 

-8.16984 

-8.15081 

-8.16912 

-8.17858 

-8.15301 

-8.1361 

-8.15632 

-8.16736 

-8.09729 

-8.12996 

-8.16886 

-8.13858 

-8.16493 

-8.1709 

-8.15097 

-8.16653 

-8.18467 

-8.17554 

-8.15894 

-8.17614 

-8.13063 

-8.17381 

-8.18347 

-8.15888 

-8.15675 

-8.16754 

-8.16584 


Sanity-Check XY 
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RESULTS 


HEATHAPS 
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RESULTS: HEATHAPS 




RESULTS: HAP BALANCE 


Gamevariantname 


Year of Da.. Week of D.. 

Team Color 

SWATnums 

2015 Week 44 

Blue 

53% 

Week 45 

Blue 

53% 

Week 46 

Blue 

54% 

Week 47 

Blue 

54% 

Week 48 

Blue 

54% 

Week 49 

Blue 

54% 

Week 50 

Blue 

54% 

Week 51 

Blue 

54% 

Week 52 

Blue 

53% 

Week 53 

Blue 

53% 

2016 Week 1 

Blue 

53% 

Week 2 

Blue 

53% 

Week 3 

Blue 

53% 

Week 4 

Blue 

53% 

Week 5 

Blue 

51% 

Week 6 

Blue 

50% 

Week 7 

Blue 

50% 

Week 8 

Blue 

50% 

Week 9 

Blue 

50% 

Week 10 

Blue 

50% 

Week 11 

Blue 

50% 


Gamevariantname 


Year of Da.. 

Week of D.. 

Team Color 

SWATnums 

2015 

Week 45 

Blue 

52% 


Week 46 

Blue 

53% 


Week 47 

Blue 

53% 


Week 48 

Blue 

53% 


Week 49 

Blue 

53% 


Week 50 

Blue 

53% 


Week 51 

Blue 

53% 


Week 52 

Blue 

52% 


Week 53 

Blue 

52% 

2016 

Week 1 

Blue 

53% 


Week 2 

Blue 

51% 


Week 3 

Blue 

53% 


Week 4 

Blue 

51% 


Week 5 

Blue 

50% 


Week 6 

Blue 

50% 


Week 7 

Blue 

50% 


Week 8 

Blue 

49% 


Week 9 

Blue 

50% 


Week 10 

Blue 

48% 


Week 11 

Blue 

49% 


RESULTS: ANIMATED HEATMAPS 




RESULTS: 


ANIMATED HEATMAPS 


RESULTS: 


■ 


ANIMATED HEATMAPS 


avgiee 

I 

| 


50 100 

X 


RESULTS: 


ANIMATED HEATMAPS 


RESULTS: 


ANIMATED HEATMAPS 


GDC 

THANK YOU! 

(AND PLEASE FILL OUT THE SURVEY) 

TOMM a MICROSOFT. con 
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